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Detailed Action 

Drawings 

Objections 

1 . The drawings are objected to because: 

a. In Figures 1-3, the reference numbers and various other captions are too light to be reproduced and 
are in some cases illegible (e.g. first functional block in Fig. 1). 

b. The dashed boxes of Fig. 3 are drawn in such a way as to render the units, which they demarcate, 
indiscernible. 

c. The lines indicating the logical flow of the JPEG image compressor depicted in Fig. 3 are difficult 
to follow. The lines near the center of Fig. 3 are particularly heinous. 

d. The formulas listed in several of the functional blocks depicted in Fig. 3 are difficult to read. For 
example, see the formula for Sy and the formula in block 118. 

e. Though Sy in Fig. 3 is apparently attributed to one of the flow lines within its vicinity, it cannot be 
determined from the drawing which one(s) of those lines Sy is actually attributed to. 

A proposed drawing correction or corrected drawings are required in reply to the Office action to avoid 
abandonment of the application. The objection to the drawings will not be held in abeyance. 

Specification 

Objections 

2. The disclosure is objected to because of the following informalities. 

a. On page 3, lines 20-2 1 of the applicant's disclosure, the applicant recites: "The difference between 
Y nin and represents lost image information causing distortion to be introduced". This 
difference is trivially 0. However, it is clear from the preceding parts of the disclosure that the 
applicant intended the following: "The difference between reconstructed Y nm and the originally 
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transformed coefficients Y™ represents lost image information causing distortion to be 
introduced". 

b. On page 8, line 25 of the applicant's disclosure the following formula appears: 

Clearly this is a typographical error. 

c. On page 9, line 1 1 of the applicant's disclosure, T ijm is used where T ijmn was intended. 

d. On page 14 of the applicant's disclosure, the convolution formula for Hy, H t j = ^ V i _ m j. m G^ is 

m,n 

incorrect. First, it is unclear as to what the indices of summation are. If the summation is 
performed over all m and n, then for H to be a convolution of V and G, the proper expression for 
the ij-th element of H is, H f j = ^^ m j- m O mn . Note also that the preceding usage of the indices, 

Vm,/i 

i and j, in the applicant's disclosed formula for Hy can be misleading. The formulas listed here and 
in the disclosure imply that Hy is the ij-th element of a 2D convolution. However, Hy is already 
used in the applicant's disclosure to represent this convolution between the Laplacian edge kernel 
G and the window Vy where the indices, ij, are associated with the ij-th image block, By. See page 
13 of the applicant's disclosure. Usage of a different set of indices to indicate an element of the 
matrix, Hy, would resolve any potential confusion. 
Appropriate correction is required. 



Claims 

Objections 

3. Claims 1-2, and 6-10 are objected to because of the following informalities. 

a. Claim 1 item (e) indicates the division of by a matrix, S min *Q. It is clear that the applicant intended 
the division by S m j n *Qmn- Claims 2, 6-8, and 10 are similarly objected to. 
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b. In claim 6, the following formula A mn = • /« (B i} ) mn > appears to be a typographic error. 

(U) 

(By ) mn , and the operation • are undefined. For the remainder of this document it will be 
assumed that the intended formula is: A mn = ^hj(^ij)mn • 

c. Claim 7, item (d) contains the formula, S a = ly (S^ - S^). Clearly, Sy = Iy (S^ - S mill ) was 
intended. 

d. Claim 7, item (f) contains the expression Q*S min > where Q*S min was intended. 

e. Claim 8 contains two items labeled (a). 

f. Item (a) of claim 8 recites "shrinking the colour channels U and V by a fraction of their size". This 
terminology is misleading. Item (a) of claim 8, as written, seems to indicate that the dimensions of 
the image, or at least the UV component thereof, are reduced. When interpreted in this manner, 
claim 8 is not consistent with the specification. However, it is clear from the disclosure that the 
applicant intended item (a) of claim 8 to indicate that the color channels U and V are subsampled 
at some fixed ratio relative to the color channel. The "shrinking factor" o f claim 9 is similarly 
objected to. 

Appropriate correction is required. 

Rejections Under U.S.C. § 1 12(2) 

4. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject 
matter which the applicant regards as his invention. 

5. Claim 6 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite for failing to particularly 
point out and distinctly claim the subject matter which applicant regards as the invention. Claim 6 recites the 
limitation "said Q". Worded as such, claim 6 has multiple antecedents (i.e. the said Q can refer to that which is 
defined in claim 1 item (i) or claim 1 item (ii)). It is unclear, in claim 6, as to whether the applicant is referring to the 
Q defined in item (i) of claim 1 or item (ii) of claiml . It is assumed subsequently in this document that said Q refers 
to the global quantization matrix Q defined in item (ii) of claim 1 . 
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Rejections Under U.S. C. 6 103(a) 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set 
forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if 
the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would 
have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1, 3, 8-10, and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable over Konstantinides et 
al. (U.S. Patent 6,314,208), in view of Yovanof et al. (U.S. Patent 5,677,689). 

8. The following is in regard to claim 1. Note that the image compression system proposed by Konstantinides 
et al. implements an image compression algorithm that is JPEG compliant (Konstantinides et al. column 2, last 
paragraph). Therefore, by adhering to the JPEG standard, the following aspects of that algorithm can be regarded as 
inherent. 

The algorithm, upon which the Konstantinides et al.'s disclosed system is based, is a method of JPEG 
compression of an image frame divided up into a plurality of non-overlapping, tiled 8x8 pixel blocks By 
where i, j are integers covering all of the blocks in the image frame. Note that Konstantinides et al. use y ; to 
denote these blocks (Konstantinides et al. column 4, last paragraph). By and y s will be used interchangeably 
throughout this document. The method comprises: 

a. Forming a discrete cosine transform (DCT) of each block y { of the image frame to produce a 
matrix of blocks of transform coefficients Dy. Note that Konstantinides et al. use Yj to denote such 
a matrix of blocks of transform coefficients (Konstantinides et al., column 5, paragraphs one and 
two). Dy and Yj will be used interchangeably throughout this document. 

b. Calculating a visual importance, Iy, for each block of the image. The activity measure Mj 
calculated for each block i can be regarded as a measure of visual importance for block i. See 
Konstantinides et al. column 5, last two paragraphs, column 6, lines 27-54, and column 7, lines 
45-56. 
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Although Konstantinides et al. do not limit the range of Mj to the interval [0,1], as the applicant does, it can 
be seen from the formula in column 6 and the pseudocode listed in column 7 of Konstantinides et al. that 
Mi > 0. Konstantinides et al. suggest a practical range for M { derived from empirical data. See 
Konstantinides et al. column 7, lines 42-44. Furthermore, note that a higher Mj suggests a higher activity, 
which, in turn indicates a block of higher visual importance. Similarly, a lower Mj suggests a lower activity, 
which, in turn indicates a block of low visual importance. This is consistent with the Iy of the applicant's 
claimed method. It would be trivial for one of ordinary skill in the art to map the suggested range of Mj to 
the interval [0,1]. 
Additionally, 

The algorithm, upon which the Konstantinides et al.'s disclosed system is based, is a method of JPEG 
compression further comprising: 

c. Forming a global quantization matrix Q (denoted Q e by Konstantinides et al.) by one of: 

i. Selecting a standard JPEG quantization matrix table. See column 8, lines 18-22 of 
Konstantinides et al. 

ii. Selecting a quantization table such that the magnitude of each quantization matrix 
coefficient Qy is inversely proportional to the importance in the image of the 
corresponding DCT basis vector. See Konstantinides et al. Figs. 2 and 3, column 7, lines 
45-51, and column 8, lines 18-22. 

d. Selecting a linear scaling factor Sjj defining bounds over which the image is to be variably 
quantized. Note that Konstantinides et al. use qsalej to denote this linear scaling factor. See 
Konstantinides et al. Figs. 2-3 and column 7, last paragraph. Sjj and qscalej will be used 
interchangeably throughout this document. 

While Konstantinides et al. do teach quantizing the transform coefficients, Y;, by an equivalent of dividing them by 
matrix element Q c \j,k] to obtain quantized transform coefficients, YQ ei [j,k] (see Konstantinides et al. column 5), 
where it is understood that Q e , in this case, is the result of scaling the original Q c by the scalars quale; (see 
Konstantinides et al. Figs. 2 and 3, and column 8, lines 25-27), they do not teach utilizing a user-selected scaling 
factor, such as S^, in the manner of item (e) in claim 1. 



Application/Control Number: 09/759,150 Page 7 

Art Unit: 2623 

9. Yovanof et al. disclose the usage of a user-selectable scalar (referred to as the Q-factor by Yovanof et al.) 
that is used to scale the quanization matrix (referred to as the Q-table in Yovanof et al.), in a manner similar to the 
S min of the applicant or the qscale ; of Konstantinides et al. See Yovanof et al. column 5, lines 15-20. It should be 
clear that modifying the compression method of Konstantinides et al. to provide for a user-selectable scalar, such as 
the one suggested by Yovanof et al, would be a trivial enterprise for one of ordinary skill in the art. Since the 
scaling factor is indirectly related to the compression ratio and directly related to the quality of the compressed 
image (Yovanof et al. column 5, lines 15-17), providing a user-selectable scalar to scale the quantization matrix, 
advantageously gives the user the freedom to manually control the inherent tradeoff between compression ratio and 
compressed image quality. Given this advantage and the simplicity of such a modification, it would have been 
obvious to one of ordinary skill in the art, at the time of the applicant's claimed invention, to utilize a user-selectable 
scalar, such as the Q-factor of Yovanof et al., in lieu of the scalar qsalei, used in the compression algorithm taught 
by Konstantinides et al, during the quantization of the transform coefficients. Taking this into account and the 
discussion above, it is clear that the teachings of Yovanof et al. and those Konstantinides et al., when combined in 
the manner just described, address claim 1, item (e). 

10. The algorithm obtained by the teachings of Konstantinides et al. and Yovanof et al., combined in the 
manner discussed above, is a method of JPEG compression further comprising: 

f. Entropy encoding quantized coefficients T ijmn and Q*S min to create a JPEG image file. See the 
discussion above, with respect to Yovanof et al.'s Q-factor. See also Fig. 3 of Konstantinides, et 
al. Note in Fig. 3 that the output of the quantizer 18 (i.e. the quantized transform coefficients, 
Y QeiU>k] - the output of multiplier 52) and the output of the scale computer 62 are both fed to the 
entropy encoder 20. 

Thus, it has been shown that the teachings of Konstantinides et al. and Yovanof et al., when combined in the manner 
discussed above, address all aspects of claim 1 . 

11. The following is in regard to claim 3. The teachings of Konstantinides et al. and Yovanof et al., as 
combined in the manner discussed above, yield a method, in accordance with claim 1, that further includes 
calculating a linear scaling factor Sy = Iy^S,™- S min ) + S min where S min and S niax are user specified to define bounds 
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over which the image will be variably quantized. See the last paragraph of column 7 in Konstantinides et al. Note in 
particular the equation for qscakj. With regard to claim 2, observe the following from this equation. 

a. qscalej is clearly linear (piecewise linear). 

b. As mentioned above, M { is analogous to Iy. 

c. a and b are user-selected, as implied by Konstantinides et al. in column 7, lines 65-67. 

d. As mentioned above, qscale ; determines the degree of quantization of the transform coefficients. 

e. a and b define the bounds of qscale; and, hence, the magnitude of quantization. 
Note that b can be trivially chosen to be S^. Furthermore, any arbitrarily chosen a can be expressed as 

a = (S^ - b) and, if b = S min , then a = (S^ - S^y. In this case, qscalej would have a form similar to the linear 
scaling factor of claim 3. 

12. The following is with regard to claim 5. Note that the image compression system proposed by 
Konstantinides et al. implements an image compression algorithm that is JPEG compliant (Konstantinides et al. 
column 2, last paragraph). It is well known in the art that the default format for color image data, according to the 
JPEG standard, is YUV (see, for example, Konstantinides et al. column 8, lines 33-36). Furthermore, the 
chrominance channels (UV) of the input image are typically subsampled with respect to luminance channel (Y). It 
should be clear that, in order to be practically applied to a YUV image, the various steps of the JPEG compression 
algorithm (e.g. DCT, quantization, etc.) must be performed each of the color channels. Therefore, in light of the 
discussion above with regard to claim 1, the combined teachings of Konstantinides et al. and Yovanof et al. address 
the following aspects of claim 8. 

A method of JPEG compression of a color image represented by channels Y for grayscale data, and U and 

V each for color, comprising 

a. Shrinking (subsampling) the color channels U and V by a fraction of their size; 

b. Forming a discrete cosine transform (DCT) Dy for each block By of each of channels Y, U and V; 



Expressing the coefficient a in this form may particularly useful. It would be apparent to one of ordinary skill in the art that, if Mj is constrained 
to be within the interval [0,1], as suggested above, then qscale; would become a linear interpolation between two user-selected values, S^and 
S max , for 0.4 ^ qscalej £ 2. 
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d. Quantizing the transform coefficients for each of the Y, U and V channels by dividing them by a 
factor Sy Q', where Sy is a linear scaling factor for each of channels Y, U and V and Q' is the 
quantization table for the associated channel being quantized; 

e. Entropy encoding quantized coefficients Ti jmn and Q^S^, where S min is a user selected minimum 
scaling factor for each of channels Y, U, and V, to create a JPEG image file for each of channels 



13. With regard to item (c), calculating a visual importance, Iy, for each Y channel block of each image and 
setting Iij=max{Iij values for corresponding Y channel blocks} for blocks in the U and V channels, note the 
following. In the compression algorithm implemented by the compression system of Konstantinides et al., the 
activity metric (visual importance) is calculated for each Y channel block of the image and applied to the associated 
UV blocks (Konstantinides et al column 8, lines 41-43). This can be trivially extended to deal with a subsampled 
image, where for each subsampled chrominance block (UV) there may be multiple corresponding luminance blocks 
(Y). Clearly, when confronted with such a situation, one of ordinary skill in the art, in an effort to preserve image 
quality, would select, among the multiple Y blocks, the block with highest visual importance (activity measure). 
One could then extend the teachings of Konstantinides et al. to subsampled images by calculating the activity metric 
for each Y block associated with the given UV blocks and applying to those UV blocks the maximum of the activity 
metrics calculated for the associated Y blocks. Konstantinides et al. also addresses item (d) of claim 8 (forming a 
global quantization matrix Q for the Y channel block and one for channels U and V combined such that a magnitude 
of each quantization matrix coefficient Qy is inversely proportional to an importance in the image of a corresponding 
DCT basis vector). This follows from the preceding discussion and the one presented above for item (c,ii) in claim 

1. Thus, it has been shown that the teachings of Konstantinides et al. and Yovanof et al., when combined in the 
manner discussed above, can be trivially extended by one of ordinary skill in the art so as to address all aspects of 
claim 8. 

14. The following is with regard to claim 9. The chrominance components of YUV images that are JPEG 
compressed are typically subsampled at a 2:1 ratio with respect to the luminance component. Given this, it would 
have been obvious for one of ordinary skill in the art to subsample the chrominance components of the input image 



Y, U and V. 
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at a 2:1 ratio relative to the luminance component in the JPEG compression method, obtained by the teachings of 
Konstantinides et al. and Yovanof et al, when combined in the manner discussed above with respect to claim 8. 

15. The following is with regard to claim 10. The apparatus of applicant's claim 10 is an implementation of the 
method of claim 1. Therefore, with regard to claim 10, arguments analogous to those presented for claim 1, are 
applicable. In addition, with regard to item (d) of claim 10, arguments analogous to those presented for claim 3 are 
applicable. 

16. The following is with regard to claim 12. The apparatus of applicant's claim 12 is an implementation of the 
method of claim 3. Therefore, with regard to claim 12, arguments analogous to those presented for claim 3, are 
applicable. 



17. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Konstantinides et al., in view of 
Yovanof et al. as applied to claim 1 above, and in further view of Memon et al. (U.S. Patent Publication 
2001/0043754A1). 

18. The following is with regard to claim 4. As shown above, the compression method obtained from the 
teachings of Konstantinides et al. and Yovanof et al, as combined in the manner discussed above, yield a method, in 
accordance with claim 1. However, in such a method, the visual importance, Iy, is not determined by discrete edge 
detection and summation of transform coefficients. 

19. Memon et al. show a JPEG compliant compression algorithm (Memon et al., paragraphs [0026] and 
[0040]) wherein a visual importance is determined by discrete edge detection and a simple activity measure 
computed for each block. See Memon et al., Fig. 4 and paragraphs [0038] and [0053]-[0060]. See also Memon et al. 
paragraphs [0013]-[0021] for a related discussion. Note there that edges are detected in the block classification 
scheme of Memon et al. The visual importance, in this case can be regarded as the classification of the block in 
combination with the simple activity measure associated with that block. Memon et al. do not show how the block 
activity measure is derived, however, it would be understood by one of ordinary skill in the art that the activity 
measure, M ; , given by Konstantinides et al. (Konstantinides et al., column 6), or the activity metric, A, given by 
Yovanof et al. (Yovanof et al., column 6), would suffice. In either case, a summation of the transform coefficients is 
computed. Modification of the compression method obtained from the combined teachings of Konstantinides et al. 
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and Yovanof et al. to incorporate the block classification taught by Memon et al. would be straightforward to one of 
ordinary skill in the art, since: (a) all three methods are based on the JPEG standard, and (b) the block classification 
would only effect the computation of the scaling factor (e.g. as in reference number 350 of Fig. 3 of Memon et al, 
reference number 62 of Fig. 3 of Konstantinides, et al., or reference number 318 of Yovanof et al.). The motivation 
to perform such a modification is that the perceptual classification scheme proposed by Memon et al. provides 
additional flexibility in adapting the quantization table scaling factor to the perceptually distinct portions of the 
image, while maintaining computational simplicity. This flexibility allows for the reduction of artifacts in JPEG 
compressed images. See Memon et al. paragraph [0026]. Given the relative ease of such a modification and its 
apparent advantage, it would have been obvious to one of ordinary skill in the art, at the time of the applicant's 
claimed invention, to incorporate the perceptual block classification scheme described by Memon et al. into the 
compression method obtained by combining, in the manner discussed above with regard to claim 1, the teachings of 
Konstantinides et al. and Yovanof et al. In doing so, one would obtain a method according to claim 1, where the 
visual importance of a block is determined by discrete edge detection and summation of transform coefficients. This 
is in accordance with claim 4. 



allowable if rewritten in independent form including all of the limitations of the base claim and any intervening 
claims 

21. Claim 6 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 1 12, second 
paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening 
claims. 



Allowable Subject Matter 



Objections, Allowable Subject Matter 



20. 



Claims 2, 5, 6 and 1 1 are objected to as being dependent upon a rejected base claim, but would be 



Allowable Subject Matter 



22. Claim 7 is allowed. 
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23. 



The following is a statement of reasons for the indication of allowable subject matter. 



24. 



The following is with regard to claim 2, 7 and J I. The combined teachings of Konstantinides et al. and 



Yovanof et al. can be shown to address item (a) of claim 2 and the rounding (Dymn / (Smin+Q)) to the nearest integer 
to form quantized transform coefficients, also of claim 2. However, item (b) is not shown in either of these 
references. Essentially, item (b) of claim 2 claims the quantization of the transform coefficients by assigning them to 
quantization levels that are integer powers of 2, when the error of this method of quantization is less than or equal to 
the error the standard quantization technique (e.g. T ijmn = D ijnui / (Sij*Q) or Y Qci [j,k] = Yi[j,k] / Q e D,k], as in 
Konstantinides et al.). It should be noted that the usage of quantization levels that are integral powers of two is not 
unique. Juri et al., for example, teach the usage of adaptive quantization within a video compression, similar in 
principle to JPEG, wherein the quantizing width of the transformed coefficients (Juri et al. Fig. 1 reference number 
4), corresponding to an 8x8 block (Juri et al. column 6, lines 25-32) of a video frame, is varied so that transformed 
coefficients corresponding to less visually important image data are quantized using a larger quantization width (Juri 
et al. Abstract). That width varies as 2\ where ieZ is the minimal number of bits required to express the 

transformed coefficient being quantized as a binary number. The quantization proposed in item (b) of the applicant's 
claim 2 accomplishes essentially the same result. However, the quantization of the transform coefficients shown by 
Juri et al. differs from that of item (b) in the applicant's claim 2, in that the applicant actually sets the quantized 
coefficient T ijmn to a value having a form similar to 2\ whereas Juri et al. divide the transform coefficient by 2' to 
obtain the quantized transform coefficient. This discussion is applicable to claim 7 and 1 1 . 

25. The following is with regard to claim 5. As shown above, in regard to claim 4, evaluating the visual 
importance of an image block based on some form of edge detection exists in the prior art. Convolution with an 
edge-tracing kernel is known in the art as a means for edge detection. However, utilizing such an edge detection 
technique and summing center 10x10 matrix values of said convolution to produce a summed value, and 
normalizing said summed value to produce a visual importance, is unique and non-obvious. 

26. The following is with regard to claim 6. While techniques for generating quantization matrices abound in 
the prior art, the applicant proposes a unique and non-obvious approach in claim 6. 
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Citation of Relevant Prior Art 



27. 



The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: 



a. U.S. Patent 5,481,309, Juri et al., "Video Signal Bit Rate Reduction Apparatus Having Adaptive 
Quantization". Juri et al. teach the usage of adaptive quantization within a video compression, 
similar in principle to JPEG, wherein the quantizing width of the transformed coefficients (Juri et 
al. Fig. 1 reference number 4), corresponding to an 8x8 block (Juri et al. column 6, lines 25-32) of 
a video frame, is varied so that transformed coefficients corresponding to less visually important 
image data are quantized using a larger quantization width (Juri et al. Abstract). That width varies 
as 2 1 , where ieZ is the minimal number of bits required to express the transformed coefficient 
being quantized as a binary number. This is similar in principle to quantization proposed in item 
(b) of applicant's claim 2 and 1 land item (e,iii) of claim 7. 

b. "An Adaptive Perceptual Quantization Algorithm For Video Coding", Chun et al., IEEE 
Transactions on Consumer Electronics, Vol. 39, No. 3, August 1993. This article shows a 
perceptual block classification scheme similar to that of Memon et al. and, therefore, addresses 
applicant's claim 4. 

c. "A JPEG Variable Quantization Method for Compound Documents", of Konstantinides et al., 
IEEE Transactions on Image Processing, Vol. 9, No.7, July 2000. This article contains a more 
detailed treatment of several aspects of the invention disclosed by Konstantinides et al. in U.S. 
Patent 6,314,208. 

d. "Information Flow if Non-Uniform Differential Quantization", Dusan Agrez, Instrumentation and 
Measurement Technology Conference, 1999. IMTC/99. Proceedings of the 16th IEEE, Volume: 
3, 24-26 May 1999 Pages: 1667 - 1672 vol.3 This article shows a non-uniform quantization 
technique employing variable quantization widths, where those widths are varies proportionally to 
a powers of two. See section 2 and Fig. 4. 
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e. "JPEG Still Image Data Compression Standard", 1 st Edition 1993, Pennebaker, W. and Mitchell, J. 
The book provides a thorough treatment of the JPEG standard. 

Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to Kevin Siangchin whose telephone number is (703)308-6604. The examiner can normally be reached on 
9:00am - 5:30pm, Monday - Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Amelia Au can 
be reached on (703)308-6604. The fax phone number for the organization where this application or proceeding is 
assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding should be directed 
to the receptionist whose telephone number is (703)306-0377. 
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